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ABSTRACT 

Two evaluative efforts sought to assess the 
educational impact of computer applications in the social science 
curriculum. Questionnaires were administered to groups of students: 
1) who either had just finished a section of a Principles of 
Sociology course at Oberlin College which dealt with computer-based 
research techniques in the social sciences; or 2) who had taken +:he 
section from one to three semesters earlier. The majority of both 
groups regarded the computer research material positively and felt it 
should be retained in the course. The skills learned were put to use 
in this and other courses, and the research curriculum increased the 
students • understanding of empirical literature. Responses to the 
questionnaires indicated that the students needed greater exposure to 
the discipline in order to handle data competently, that such 
exposure should be coupled with practical experience in empirical 
research projects, and that ways need to be developed to utilize in 
upper level courses the skills learned in the introductory course. 
Nevertheless, the overall conclusion was that the computer-based 
research curriculum was an important and well-received component of 
an introductory sociology course. (Author/LB) 
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As the. incor pcraticn of the computer into the undergraduate social science curriculum 
has become more widespread in recent years, the need for continuous and systematic evaluation 
of these endeavcrs has likewise increased. Perhaps the most straightf orwaird statement to 
this effect is found in the title cf a paper presented by M€yers[1] at the second Conference 
on Computers in the Undergraduate Curricula, "We Don't Know What We Are Doing.'* Although his 
assertion is probably less valid new as the accumulating evidence [1], [2], [3], begins to 
point to some salutary effects cf computer utilization in the social science curricula, the 
need for additional evaluative data persists. 

CD 

^m^i Moreover, there are other questions yet to be posed. However much we know about what we 

^ are doing, we surely know next to nothing about what we have done. This is more than a play 
' ' on words; rather, it speaks to the necessarily myopic quality of the typical assessment 
CO instrument — "on-the-spot" course evaluations. While immediate fe^idback is often helpful in 
C3> providing insight into student reaction to our computer-based curricular innovations, there 
is an equally important set of questions which on-the-spot evaluation cannot answer. How is 
a the computer experience viewed from the perspective of hindsight? Does exposure to computer 
I i I skills in an introductory course provide an important additional dimension of preparation for 
other courses within the same or in related disciplines? To what extent have the computer 
skills learned in an introductory course been used in other courses?. 

In other words, both short-run and long-term evaluative questions must be asked in order 
to assess the educational impact cf computer applications in the social science curricula. 
The present paper is concerned with both types of questions. In the context of an 
introductory sociology course which har a computer-based research component as an integral 
part, two. sets of evaluations are analyzed: (a) evaluations submitted by students upon 
completion cf the course; and (b) retrospective evaluations from students who took the course 
from one tc three semesters previously. 



Zjg j_Pggg£iR^j^gIl-Pf _y-l^g--?Q'^F_gg__A^j^ ^he Ccmpu ter Curriculum 

Average enrollment in the Principles of Sociology course at Oberlin College is H5 
students per section with three to four sections available each semester. The course itself 
is divided into two parts. In the first half, students are introduced to the basic 
principles of sociology through lectures, readings, and class discussion. A student may 
elect to take this segment only and, upon successful completion, receive two hours of credit 
as well as prerequisite credit for many upper level courses. For students who elect to take 
the entire course (typically two-thirds to three-quarters cf the original enrollment) , the 
emphasis in the second part is on the application of the principles to individual projects. 
In my own sections, the computer-based research curriculum is introduced at the beginning of 
the second part of the ccurse[5]« 

The research curriculum takes five 65-minute class sessions: 

1« The first session provides students with a brief and general introduction to 
research methods. Topics included are the reciprocal relationship between theory 
and research, a comparative analysis of sever a 1 ma jor resea rch strategies, and a 
discussion of sampling* Arrangements are also made for stadentc to receive 
instruction in keypunching . 
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2* In the second session, detailed consideration is given to the structure of data 
tv» sets, the use cf codebocks, the mechanics cf using EAFPACK to replicate a major 

' finding taken from the sociological literature enploying one of our own data sets, 

CO and job submission procedures. Students are assigned the submission of a pre- 

selected run which is to be discussed in the third session. 



D 



3. This class period is used to examine carefully the output from the assigned run. 
Because the output invclves a bivariate relationship, we discuss the proper 
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direction ^ in vhich tc percentage a table, the substantive nature * of th*? 
relationship, the use and coauing of obst:2rved and expected frequencies, and chi- 
square as a test of significance. in prepacation for the following session 
consideration is given to the mechanics of controlling for a third variable and to 
procedures for recoding and collapsing categories. The assignment for the 
following session in the resubmission of the original run with' the addition of i 
theoretically relevant control variable. 

^. In the fourth session, consideration is given to the process cf elaboration and to 
the various ways in which the relationship between twc variables' might be modified 
when a control variable is used. The assigned output is then examined to apply th^ 
principles of elaboration. In preparation for the next session, students are givon 
instruction in obtaining univariate frequency distributions and are.^assigned a 
problem requiring the frequency distributions for several variables. 

^. In the. final session, the several uses of univariate, distributions are discussed ir 
conjunction with the assigned output* Also considered is the application of 
CAFPACK to either data sets, the use of multiple control variables, and several 
other options available in DAFPACK[6]. 

?» 

Toward the end cf the research curriculum, students are given a catalogue containing 
brief descriptions of the approximately twenty-five data sets available for their use. They 
are asked to submit a statement of a proposed research project involving either computer* 
based data analysis or the more traditional library research. On tho basis of these 
proposals, the students' are assigned to small groups, comprised of students with similar 
substantive interests^ which meet weekly for progress reports and discussion of their 
individual projects. A term paper, reporting the results of their research, is required of 
all students. 



The evaluation of this computer-based research curriculum in" the introductory sociology 
course is derived from two sets of data. The first data source is a series of 
questionnairest 7 ] given tc the students in my section of the introductory sociology course- in 
the Fall cf 1972. Information is available^ from 25 of the 28 students who were enrolled for 
the full four hours. 

The second s^t of data comes from a course evaluation questionnaire sent to students who 
were enrolled in my sections of the course during the "(jhrea previous semesters in which the 
coiHputer curriculum was offered. No attempts were made to contact students who had 
graduated, withdrawn from college, or enrolled in off-campus study programs. Thus, of the 78 
students enrolled for the full four hours, 58 were eligible and returns were received, after 
one follow-up, from ^2 students or 72% .of the eligible sample. 

Sxaluati^ons^From_Current_St 

Fr om I several perspectives, the data indici'te' that the majority of students viewed the 
computer-based research curriculum in positive terms. In response. to the question "Was the 
two week/ pe.riod spent cn research methods valuable to you," l^% of the 26 students answered 
in the affirmative. Illustrative of the favorable comments are the following: 

Now I know how to get data from any of the data sets Oberlin has. I have learned 
something about the way computers work. 

It gave one at the least a minimal working familiarity with using the computer in 
research and also an ability to read and^understand the tables and references concerning 
research in various periodicals and books. * 

On the negative side, the following comments are typical of those made by students who did 
not think' that the computer curriculum was valuable to them: 

■* " " 

I couldn't understand it very well. 

I didn^t utilize the knowledge any further and I don't think that I will. 

Quite apart from the personal value of the computer curriculum, the students were asked 
whether the se.ssions on research methods should be retained in future offerings of the 
course. The overwhelming majority indicated that the research curriculum should remain a 



part of the course with only disagroeing. However, while most cf the students favored 
retention of this comfcnent of the course, 63^ felt that the research curriculum should be 
cade optional while 33% thought it should be required. Two theires consistently appeared in 
the qualitative responses'. On the one hand, several students coanjented that where a choice 
is given between working cn an empirical research project and a library research project, the 
computer curriculum is apt to be far less meaningful and unimportant to persons who select 
the latter. 

On the other hand, those who indicated thal^^the computer curriculum should be required 
often noted that apprehension would cause many stu^J-ents to oiss a good experience if it were 
not required. 

One further bit of evidence supports this notion that required exposure to the computer 
curricrulum facilitates a positive attitude toward the experience. In each of the three 
qu Jtionnaires administered during the term, the students were asked whether they thought 
"that an elementary introduction to ccmpu ter-based methods cf data analysis should be part of 
an introductory sociology course." At the very beginning cf the course, only 31% of the 
students enrolled for the full four hours responded "yes." At the end of the first part of 
the term, the proportion saying yes had increased slightly tc 35%. At the end of the second 
part of the term, however, 60% of the students agreed that snch an experience should be part 
of an introductory sociology course. 

Not only is the eva luation cf the computer ci:rriculum predominantly positive, but there 
are also indications that the majority of students expect tc use these skills in a variety of 
ways in other courses. Thus, 62% of the students answered in the affirmative to the question 
••Do you plan to take any aore courses or do work in ether courses that will allow you to make 
use of these skills." 

The data discussed to this point suggest a generally favoreble response to the computer- 
based research curricula:?, itself. More detailed analysis of tht data, however, shows that 
such responses are associated with whether the student made further use of these skills in 
the course by undertaking an empirical research project. The data in Table 1 indicate that 
students who pursued a computer project involving secondary analysis of data rather than a 
library research project were more likely to say that the tvio week period spent on research 
tnethods was valuable, that these sessions ought to te retained, that an elementary 
introduction to ccmputor-tased methods of data analysis should be part of an introductory 
sociology course, and that they plan to utilize these skills in other courses. Students who 
conducted empirical projects were more likely to say that the sociology course was the best 
of the introductory courses they had taken, and they also planned to take more courses in the 
department than did those who pursued a library research prcject. Finally, in terms of both 
what they learned from their research project and their interest in it, a larger proportion 
of those doing computer work stated that their projects were both very valuable and very 
interesting. Perhaps the most enthusiastic (and gratifying) comments were those relating to 
the students' evaluations of their empirical research projects: 



TABLE 1 

comparison of evaluatioas from students doing computer-based 
empirical projects and students doing litrary research projects 



S-IIS: Was the tvo week period spent on research 
methods valuable to you? 

^.RETAIN: Should these sessions cn research methods 
be retained in the future? 

Should an elecentary introduction to computer- 
based methods of . da ta "^analysis be part of an introductory 
sociology course? 

?-.IES: Do you plan to take any more courses or do work 
in other courses that will allow you to make use of 
these skills? 

S-IILHAS_BEE]i_TH.E_BEST : How did Sociology 10 1 stack up 
against other introductory courses you've taken? 

Beyond Sociology 101, how much more sociology do ycu 
expect to take? 

None, probably 

Maybe one or twc more courses 

Certainly plan to take some ether courses 

Thinking about m.-ijoring or at least minoring in 
sociology 

5_YERY_ VALUABLE: In terms of what you learned from it, 
how would you characterize your individual project? 

VERY^IMTERESTING: In terms of your interest in it, 
how would you characterize your individual project? 

Number of students 



Empirical 
Pro jects 
Q3% 



13% 

27% 

0% 

36 
18 

91% 
<11) 



Library 
Projects 
62% 



92% 
50% 

39% 

3% 

21% 
i*3 

29 
7 

79% 

79% 

(15) 



I have gained valuable experience in using empirical data to subs tantiate hypotheses • • 
I also learned hew to make use of a computer system that will give me access to data 
for ether proje^s» 

I really like running data on the computer and trying to predict my results . . • 1 
really got involved in the problems I was studying and now I hope to do more research on 
them * 



1X^1 uat ion s_Pr emigre vious_Stud en 

The retrospective evaluations of the ccmputer-bas €d research curriculum provided by 
previous students are very much in accord with these of the current students. when asked 
whether the two week period spent on research methods was valuable, 76% of the ^2 students 
indicated that it was. The reasons cited were auite varied: 

It was fun. I enjoyed being able to 'Experiment on the computer. It gave me a much 
clearer understanding of some of the basics- of sociological research techniques. I also 
helped me understand the charts and statistics I encountered in my reading. 

If nothing else, it changed my opinion that sociology is nothing but book after book of 
fancy but meaningless rhetoric. Also,. I believe everybody should have some kind of 
exposure to the computer . * * like medicine, it ^ s good for you. 



Here again, however, responses varied by whether the student pursued an empirical research 
O .3ject or a library research project. Of those who conducted a computer-based secondary 



analysis of one of our data sets, 91S (N = 23) said that the research curriculum was 
valuable, while 56% (N = 19) of those whoso term projects Mere based on library research 
indicated that the curriculum was valuable. Thus, a- clear oajority of both groups of 
students view this experience in r€trospe*ct as having been valuable although it would appear 
to be considerably mere so for students who utilized the skills in their term projects. 

As was the case with the current students, most of the former students (93% of the total 
group; 96% of the students doing empirical projects and S9% of those doing library research 
projects) suggest that the research curriculum should be retained in future offerings of the 
course. However, indicated that the curriculum should be offered on an optional basis 

with only 32% saying that it should be required of all students. Those suggesting that the 
sessions should be optional generally pointed to differences among students in their 
backgrounds and interests with the implication that the course should be sufficiently 
flexible tc accommodate such diversity; 

Some people have had iJ' i x p'-^ienc e pieviously and probably don/ t need an i ri • j. ouui: I .loi; 
again. Others might prefer to ue exposed later, and probably will be more positive 
about the rest cf the course if not forced to handle research methods. 

At the same time, students arguing that the computer-based curriculum should be .a required 
component of the course noted several beneficial results regardless of studebt interest: 

It really helps you understand what you are reading. 

Empirical research is playing an increasingly important role in practically all fields 
and therefore I feel it is important to have an exposure tc it. 

The former students were also aisked a series of questions about whether they have had an 
opportunity to use the computer and empirical skills in other courses or whether they planned 
to take any courses or do work in other courses that would allow them to make use of the 
skills. In terms of actual use, 21% of the 42 students indicated that they had made use of 
these skills in other courses, and all but one of these students expected to make further use 
of them in additional courses. Of the students who had not used the skills, m% planned to 
take courses in which the computer and empirical techniques acquired in the introductory 
sociology course could be used. When actual use and expected use are considered together, 
fully 50% of the previous students either have used or still expect to use the computer-based 
empirical skills in their work in other courses. 

The actual uses to which these skills have been put range wid^.ly. Some students said 
that the general empirical skills they acquired had served them veil in understanding 
empirical data they read. Others noted that the computer experience itself yielded an 
additional dimension of preparation for other courses. Still others ji.ted specific courses 
they intended to take, such as research methods, or particular empirical studies they planned 
to conduct in the context of upper-level substantive courses. It would appear, therefore, 
that there has already been some carry over and that the potential for even more is clearly 
in evidence. 



2i§2Mssion 

The data presented in ""-he previous sections indicate that this computer-based research 
curriculum in the introductory sociology course is generally viewed in highly positive terms. 
The majority of students consider the curriculum to have been valuable and virtually all 
students are in favor of its retention in the course. Both the specific computer skills and 
the more general empirical skills have been used by some students in other courses and many 
more expect to make use of the skills. Perhaps most important are the comments of several 
students that the research curriculum has provided them with a greater facility to comprehend 
the social science literature they encounter throughout their course work. In short, the 
results point to the conclusion that a computer-based research curriculum is an important and 
well-received component of an introductory sociology course. Nevertheless, several problem 
areas apparent from the evaluations and from instructional experience with the curriculum 
merit brief mention* 

A first problem is one which did not appear in the course evaluations but which has been 
very much in evidence in the small group discussions. Although the students are able to 
learn computer skills readily and are able in short order to use tht computer to generate 
data without any real difficulty, what is more problematic, and particularly in the case of 
introductory students, is insuring that they have sufficient exposure to the discipline to 
''ci^^^ them to grasp the essentials of data presentation, discussion, and interpretation. 




it has proven extremely useful to have several carefully selected empirical articles 



availdblG as models. The provision of such exampl€s has repeatedly b€en instrumental in 
reducing student frustration at the final stages of data analysis. 



A second problem is suggested by the evaluations. It would appear that the fullest 
potential of a computer-based research curriculum is not met unless such exposure is coupled 
with an opportunity to develop and to utilize these skills through term projects involving 
empirical research. This, however, raises a very familiar and f undamerital question. 
Although the majority of students expressed the preference that the research curriculum 
itself be an option, there was ample testimony that many students would not have availed 
themselves of ..this opportunity had it not been a requirement. Likewise, there is the 
possibility that requiring an empirical paper might elicit even more interest. At issue 
here, of course, is the more general question of whether one wishes to adopt an interest 
Dodel or a prescriptive model. 

A final question relates to the optimal manner in which the skills acquired in the 
beginning course can be utilized in upper level courses. Tc date, most of our energies have 
been focused on the introductory course, and little has been dene explicitly to promote the 
option of computer-based empirical research in our upper level substantive courses. Given 
both the positive student response and the existence of a sufficiently wide range of data 
sets with variables appropriate tc almost any course in the department, it is clear that 
future efforts should be directed at ways to coordinate and tc formalize the extension of 
computer-based empirical opportunities throughout the departmental curriculum. 
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5. A detailed description of the actual computer curriculum and its rationale has been 
presented earlier [4]. This section, therefore^ will present only a very brief overview 
of the essentials of the curriculum. 

6. In addition to the class discussions, a number of documents (e.g., "The Relationship 
Between Theory and Research," "An Introduction tc D^FFACK," "Chi-Square , " "On 
Constructing and Reading Tables") have been prepared for distribution to the students 
for further guidance and reference. 

7. These questionnaires were administered on the first day of classes, at the end of the 
first part of the term, and at the end of the second part of the term. 
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